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ABSTRACT 


Newly  emerging  functional  area  models  designed  to  simulate  the  activities  of 
individual  battlefield  operating  systems  are  generating  challenging  new  validation  issues 
for  Army  analysts.  From  database  to  output,  these  new  models  require  testing  against 
real  system  performance  to  insure  no  significant  disparities  exist.  The  Command  and 
Control  Network  (C2NET)  model  is  a  prototype  Command  and  Control  Functional  Area 
Model  (C2FAM)  and  exemplifies  this  validation  challenge.  This  paper  examines  the 
C2NET  database,  input  distributions,  and  linkage  mechanism  with  a  force  evaluation 
model  as  part  of  C2NET’s  continuing  validation  effort.  The  nonstationary  Poisson 
process  is  examined  and  used  to  develop  hypotheses  about  input  distributions  linking  the 
evaluation  model’s  tactical  scenario  with  C2NET’s  input  parameters  and  existing 
database.  Fuzzy  set  theory  is  then  examined  with  applications  for  using  C2NET  output 
as  input  for  an  evaluation  model.  Areas  for  further  research  are  discussed. 
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INTRODUCTION 


Now  if  the  estimates  made. . .before  hostilities 
indicate  victory  it  is  because  calculations  show  one's 
strength  to  be  superior  to  that  of  his  enemy;  if  they 
indicate  defeat ,  it  is  because  calculations  show  that  one 
is  inferior.  With  many  calculations,  one  can  win;  with 
few  one  cannot.  How  much  less  chance  of  victory  has  one 
who  makes  none  at  alll  By  this  means  I  examine  the 
situation  and  the  outcome  will  be  clearly  apparent.  (Sun 
Tzu,  1963,  p.  71) 


The  requirement  for  valid  models  to  forecast  and  evaluate 
military  capabilities  is  certainly  no  less  today  than  it  was 
for  Sun  Tzu  in  500  B.C..  From  force  sizing  to  weapon 
procurement  strategies,  the  Army  of  the  1990 's  is  dependent  on 
land  combat  models  for  developing  a  viable  force  structure  for 
the  future  under  increasingly  tight  fiscal  constraints.  The 
family  of  land  combat  models  (Figure  1)  currently  in  use  and 
under  development  is  characterized  by  a  hierarchy  of 
evaluation  models  supported  by  functional  area  models. 

As  implied  in  their  titles,  evaluation  models  represent 
combat  operations  over  the  range  of  force  structure,  from 
individual  weapon  systems  to  theater  operations.  The 
functional  area  models  typically  abstract  the  consequences  of 
combat  operations  and  concentrate  on  the  activities  of  a 
single  battlefield  operating  system.  Output  from  a  functional 
area  model  can  stand  alone  using  acceptable  measures  of 
performance  (for  example,  when  comparing  different  systems 
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with  similar  functions) .  Ideally,  however,  the  output  from  a 
functional  area  model  is  used  as  input  for  an  evaluation 
model.  Two  immediate  and  obvious  benefits  result  from  this 
linkage;  first,  the  evaluation  model  better  approximates  the 
"reality"  of  combat,  and  second,  a  stronger  comparison  is 
available  for  analyzing  the  results  of  changes  made  in  a 
functional  area  system  through  measures  of  force  effectiveness 
(MOFE)  .  For  example,  a  measure  of  performance  (MOP)  for  a 
given  system  might  be  "processing  time"  where  System  A 
demonstrates  a  10%  improvement  over  System  B.  A  stronger 
conqparison  is  made  when  run  through  an  evaluation  model,  with 
an  MOFE  of  Red/Blue  Casualties  and  System  A  shows  a  10% 
increase  in  the  loss  exchange  ratio  over  System  B. 


Figure  1.  Family  of  Land  Combat  Models 
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The  Command  and  Control  Network  (C2NBT)  is  a  new  member  of 
the  family  of  Command  and  Control  Functional  Area  Models 
(C2FAM) .  C2NET  was  built  In  1991  as  a  direct  result  of  the 
need  for  a  C2FAM  to  interact  with  Division/Corps  Evaluation 
Models  (specifically  with  Vector- in- Commander  [VIC]  and  its 
eventual  successor,  EAGLE) .  C2NET  is  a  low  resolution  model 
representing  essential  C2  tasks  performed  by  Corps,  Division, 
and  Brigade  command,  control,  communication,  and  intelligence 
(C3I)  systems.  The  C2FAM  concept  of  desired  relationships 
between  issues,  data,  and  models  is  shown  in  Figure  2. 
Pertinent  details  regarding  the  C2NET  model  are  at  Appendix  A. 


Figure  2.  C2FAM  Concept 


As  a  recently  developed  model,  many  basic  issues  must  be 
addressed  prior  to  acceptance  of  C2NET  as  a  valid  model. 
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These  validation  issues  are  in  two  basic  categories;  model 
input  and  model  output  validation.  Model  input  encompasses 
the  model  database,  distributions,  and  linkage  from  the 
evaluation  model  into  C2NET  through  abstraction  of  the 
tactical  scenario.  Model  output  considerations  include 
selection  of  appropriate  C2  performance  parameters  (producible 
by  C2NET  and  usable  by  the  evaluation  model  to  approximate  C2 
effects  on  force  effectiveness)  and  development  of  a 
methodology  to  obtain  these  parameters  (the  model  post¬ 
processor)  . 

C2NET  in  its  current  embryonic  state  uses  subject  matter 
expert  (SME)  point  estimates  to  approximate  the  minimum, 
maximum,  and  mode  for  key  C2  task  frequencies  and  task 
durations.  While  a  valid  technique  in  model  development,  it 
fails  to  simulate  the  underlying  theoretical  distributions  (if 
present)  and  does  not  provide  for  linkage  between  the  tactical 
scenario  and  task  generation  within  C2NET.  Analysis  of 
available  empirical  data  yields  valuable  insights  about  the 
nature  of  the  distributions  in  question,  allowing  us  to 
develop  and  test  hypotheses  that  better  approximate  the 
behavior  of  C2  systems.  The  central  concept  behind  these 
hypotheses  is  the  nonstationary  (or  nonhomogeneous)  Poisson 
process.  This  process  allows  for  the  possibility  that  arrival 
rates  may  vary  as  a  function  of  time,  an  essential  attribute 
when  linking  the  tactical  scenario  (operational  tempo)  into 
C2NET.  We  expect  the  arrival  rate  for  most  C2  tasks  to 


4 


increase  as  the  operational  tempo  (OPTEMPO)  increases  (for 
example,  we  should  observe  a  low  number  of  spot  reports  when 
a  unit  is  not  in  contact,  with  a  dramatic  increase  in  spot 
reporting  during  combat) .  Examining  empirical  data  observed 
during  periods  of  "high, "  "medium, "  and  "low"  operational 
tempo  will  allow  us  to  identify  underlying  theoretical 
distributions  linked  to  the  tactical  scenario  and  finally  test 
these  distributions  using  parameters  from  the  existing 
database.  The  end  result  of  this  process  is  a  C2NET  model 
sensitive  to  the  activities  within  the  evaluation  model  and 
better  approximating  real  world  performance  of  command  and 
control  systems. 

Selection  of  reasonable  and  useful  C2  performance 
parameters  for  inclusion  into  the  evaluation  model  is  a  harder 
problem.  How  do  we  take  the  output  from  the  highly  stochastic 
C2NET  and  develop  input  for  the  totally  deterministic 
evaluation  model?  The  current  solution  is  to  manually 
calculate  single  values  for  each  echelon  in  maneuver  delays 
(required  time  spent  in  "dwell  points"  for  moving  forces, 
approximating  the  delays  observed  in  C2NET  for  the  orders 
process)  and  artillery  fire  delays  (approximating  the  lag  time 
observed  in  the  C2NET  model  in  response  to  requests  for  fire 
support) .  Clearly  command  and  control  effects  are  not  just 
delay  times  in  maneuver  and  artillery  fire,  but  approximating 
other,  less  tangible  effects,  has  thus  far  been  resistant  to 
solution  using  conventional  modelling  techniques. 
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For  example,  how  do  we  efficiently  approximate  the  command 
and  control  measures  observed  in  C2NET  for  "Artillery 
Effectiveness"  such  that  the  results  are  usable  in  an 
evaluation  model?  Simply  defining  "Artillery  Effectiveness" 
in  this  context  is  a  difficult  task;  what  factors  should  be 
included  in  our  definition  and  at  what  weight?  A  solution  to 
the  C2NET  output  dilemma  can  be  found  in  the  theory  of  fuzzy 
sets.  Briefly,  fuzzy  set  theory  allows  for  "shades"  of 
effectiveness  beyond  the  crisp  (binary)  yes  or  no  answer  to 
the  question  of  C2  performance  in  conventional  modelling. 
Applying  fuzzy  set  theory  to  our  "Artillery  Effectiveness" 
example,  we  may  allow  a  spectrum  from  "poor"  to  "excellent" 
and  use  C2NET  output  to  classify  effectiveness  for  each  unit 
by  time  period.  In  the  evaluation  model,  these  variables  are 
then  used  to  determine  not  only  delay  factors,  but  adjustments 
to  fire  power  effectiveness  (through  reductions  in  attrition 
coefficients) .  This  proposal  and  its  underlying  concepts 
receive  a  more  detailed  treatment  in  Chapter  III,  but  by  way 
of  introduction,  we  see  it  is  possible  to  approximate  abstract 
concepts  efficiently  in  an  evaluation  model. 

Finally,  as  Payne  writes,  "Another  aspect  of  the  review 
processes  necessary  to  the  validation  of  the  family  of  models, 
or  its  individual  members,  is  the  recognition  that  no  finding 
of  validity  through  review  is  final  or  permanent."  (1989, 
p.279)  .  This  must  be  especially  true  for  a  new  model  (C2NET) 
as  a  member  of  a  new  family  of  models  (C2FAM)  .  Additional 
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hypothesis  testing  will  require  a  significant  effort  in  data 
collection  through  observation  of  C2  systems  during  training 
or  operational  deployments.  Acceptance  of  fuzzy  sets  as  C2 
performance  measures  will  generate  additional  requirements  for 
SME  input  and  testing.  Implementation  of  changes  in  model 
input  parameters  will  entail  recoding  significant  portions  of 
C2NET  and  subsequent  sensitivity  analysis.  Where  appropriate, 
methodologies  to  implement  future  work  are  identified  and 
discussed  in  Chapter  IV. 

At  a  minimum,  the  analysis  of  C2NET  input  and  output 
parameters  will  provide  a  guide  to  future  validation  efforts 
and  highlight  the  current  potentials  for  significantly 
increasing  model  fidelity. 
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II 


COMMAND  AND  CONTROL  NETWORK  INPUT  VALIDATION 


Validation.  Checking  that  the  simulation  model, 
correctly  implemented,  is  a  sufficiently  close 
approximation  to  reality  for  the  intended  application.  As 
already  indicated,  no  recipe  exists  for  doing  this.  Due 
to  approximations  made  in  the  moael,  we  know  in  advance 
that  the  model  and  the  real  system  do  not  have  identical 
output  distributions;  thus,  statistical  tests  of  model 
validity  have  limited  use.  The  real  question  is  the 
practical  significance  of  any  disparities.  (Bratley,  Pox, 
and  Schrage,  1987,  pg.  8) 


The  first  step  in  validating  the  C2NET  model  is  analysis 
of  the  model's  existing  database  and  input  distributions. 
What  approximations  have  been  made  in  the  model's  input 
parameters?  Next,  using  empirical  data  and  knowledge  of  the 
system,  we  identify  improvements  that  will  reduce  disparities 
between  the  model  and  real  system  input.  For  C2NET,  this  step 
involves  identification  of  the  underlying  theoretical  input 
distributions  with  appropriate  parameters  as  well  as  a 
methodology  to  link  C2NET  with  the  tactical  scenario  of  the 
evaluation  model.  Finally,  we  must  test  our  new  construct 
against  empirical  data  of  real  world  system  performance  and 
analyze  the  results  for  robustness  (is  our  new  model  resistant 
to  inaccuracies  in  the  database?}.  The  end  result  of  this 
validation  of  C2NET  input  parameters  is  a  more  robust  model 
better  approximating  real  C2  systems  with  a  strong  linkage 
mechanism  from  the  evaluation  model,  allowing  C2NET  to  emulate 
events  in  the  tactical  scenario. 
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A.  ANALYSIS  OF  THE  CURRENT  C2NET  INPUT  PARAMETERS 

The  C2NET  database  was  developed  by  the  C2  Analysis  Cell 
(Combined  Arms  Center  [CAC] ,  Fort  Leavenworth) .  The  database 
consists  of  frequencies1,  completion  times,  and  the  personnel 
required  to  complete  key  C2  tasks.  An  argument  for  the 
heuristic  approach  used  to  develop  the  database's  estimates  is 
proposed  by  Law  and  Kelton: 


In  some  simulation  studies  it  may  not  be  possible  to 
collect  data  on  the  random  variables  of  interest,  ....  For 
example,  if  the  system  being  studied  does  not  currently 
exist  in  some  form,  collecting  data  from  the  system  is 
obviously  not  possible.  This  difficulty  can  also  occur 
for  existing  systems,  if  the  number  of  required 
probability  distributions  is  large  and  the  time  available 
for  the  simulation  study  prohibits  the  necessary  data 
collection  and  analysis. 

Let  us  assume  that  the  random  quantity  of  interest  is 
a  continuous  random  variable  X.  It  will  also  be  useful  to 
think  of  this  random  variable  as  being  the  time  to  perform 
some  task,  e.g.,  the  time  required  to  repair  a  piece  of 
equipment  when  it  fails.  The  first  step... is  to  identify 
an  interval  [a,b]  (where  a  and  b  are  real  numbers  such 
that  a<b)  in  which  it  is  felt  that  X  will  lie  with 
probability  close  to  1;  that  is,  P(X<a :  or  X>b)~ 0.  In 
order  to  obtain  subjective  estimates  of  a  and  b,  "experts" 
are  asked  for  their  most  optimistic  and  pessimistic 
estimates,  respectively,  of  the  time  to  perform  the  task. 

In  the  triangular  approach,  the  experts  are  also  asked 
for  their  subjective  estimate  of  the  most  likely  time  to 
perform  the  task.  This  most  likely  value  c  is  the  mode  of 
the  distribution  of  X.  Given  a,  b,  and  c,  the  random 
variable  X  is  then  considered  to  have  a  triangular 
distribution  on  the  interval  [a,b]  with  mode  c.  (Law  and 
Kelton,  1991,  pg.  403) 


1  In  C2NET,  estimates  for  task  frequency  have  been 
converted  to  estimates  of  interarrival  times,  e.g.,  a  task 
with  MIN/MODE/MAX  estimates  of  task  frequency  as  3/5/10  would 
have  MIN/MODE/MAX  interarrival  estimates  of  6/12/20. 
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Two  studies  (the  C2  Responsiveness  Analysis  and  the 
Division  Command  Post  Study)  have  been  completed  with  the 
triangular  distribution  database  driving  task  generation  and 
completion  within  C2NET.  These  studies  successfully 
demonstrated  C2NET's  ability  to  approximate  command  and 
control  effects  in  an  evaluation  model.  However,  they  also 
highlighted  the  necessity  of  a  link  between  the  tactical 
scenario  and  C2NET.  Without  this  linkage,  at  best  C2NET  can 
only  provide  an  aggregate  measure  of  C2  effectiveness  over  the 
life  of  the  scenario.  The  resulting  loss  of  information  (due 
to  aggregation)  is  a  potential  source  of  significant  error 
when  comparing  command  and  control  alternatives. 

Another  source  of  error  encountered  using  the  database  in 
its  current  form  is  the  approximation  made  with  the  triangular 
distribution.  A  task  with  arrivals  behaving  according  to  a 
Poisson  process  in  the  real  system  (interarrival  times 
exponentially  distributed)  and  modeled  with  a  triangular 
distribution  will  not  give  us  am  accurate  picture  of  real 
system  performance.  Compounded  over  many  tasks,  the 
disparities  in  modeled  amd  real  world  system  performamce  may 
lead  us  to  inappropriate  conclusions  when  comparing  C2  systems 
using  C2NET.  A  graph  of  a  triangular  distribution  contrasted 
with  an  exponential  distribution  plot  is  at  Figure  3,  and 
highlights  the  disparity  between  these  distributions. 
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Figure  3.  Triangular  and  Exponential  Distributions 
B.  SELECTION  OF  APPROPRIATE  INPUT  DISTRIBUTIONS 


The  first  step  in  selecting  a  particular  input 
distribution  is  to  decide  what  general  families- -e.g. , 
exponential,  normal,  or  Poisson- -appear  to  be  appropriate 
on  the  basis  of  their  shapes,  without  worrying  (yet)  about 
the  specific  parameter  values  for  these  families. 

In  some  situations,  use  can  be  made  of  prior  knowledge 
about  a  certain  random  variable's  role  in  a  system  to 
select  a  modeling  distribution  or  at  least  rule  out  some 
distributions;  this  is  done  on  theoretical  grounds  and 
does  not  require  any  data  at  all.  For  example,  if  we  feel 
that  customers  arrive  to  a  service  facility  one  at  a  time, 
at  a  constant  rate,  and  so  that  the  numbers  of  customers 
arriving  in  disjoint  time  intervals  are  independent,  there 
are  theoretical  reasons ...  for  postulating  that  the 
interarrival  times  are  IID  exponential  random  variables. 
(Law  and  Kelton,  1991,  pp.  356-7) 
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1.  Mature  and  Role  of  C2  Tasks 

Prior  to  identifying  the  general  families  of 
distributions  appropriate  for  modeling  C2  task  arrival  and 
completion  times,  we  need  to  investigate  the  nature  and  role 
of  these  tasks  in  the  C2  system.  In  general,  each  key  C2  task 
can  be  classified  into  one  of  three  categories.  These 
categories  are  defined  by  their  roles  in  the  C2  system  and 
corresponding  attributes.  The  subjective  categories  are 
"dynamic,"  "periodic,"  and  "interactive."  Common  to  all 
tasks,  regardless  of  category,  is  the  fact  that  tasks  arrive 
one  at  a  time.  For  example,  even  with  advances  made  in  multi¬ 
channel  communication  systems  we  still  expect  to  see  only  one 
spot  report  or  request  for  fire  arrive  at  any  moment  in  a  real 
C2  system.  A  list  of  key  tasks  identified  by  category  and 
battlefield  operating  system  (BOS)  is  provided  in  Appendix  A. 

"Dynamic"  refers  to  tasks  which  model  activities  with 
arrival  rates  dependent  on  the  tactical  scenario.  For 
example.  Process  Intelligence  Spot  Report  (PIS)  is  a  dynamic 
task  generated  by  observation  of  enemy  forces  or  activity.  In 
this  example,  a  spot  report  is  submitted  as  a  result  of  newly 
observed  enemy  activity  and  is  not  dependent  on  previous  enemy 
activity.  As  more  enemy  forces  come  into  contact  with 
friendly  units,  additional  spot  reports  are  generated.  We 
conclude  that  the  number  of  spot  reports  generated  in  any 
given  time  period  is  dependent  on  the  level  of  enemy  activity 
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and  independent  of  any  previously  generated  spot  reports.  In 
general  we  expect  that  for  any  dynamic  task  we  will  see 
arrivals  one  at  a  time,  at  a  rate  varying  with  the  OPTEMPO  of 
the  appropriate  BOS,  and  with  the  number  of  arrivals  in 
disjoint  intervals  independent. 

Periodic  tasks  model  activities  generated  both  as  a 
result  of  events  occurring  in  the  tactical  context  and 
according  to  standard  operating  procedures  (SOP) .  The  Update 
Status  of  Friendly  Forces  (UFF)  task  is  an  example  of  a 
periodic  task.  In  the  real  system,  updates  of  this  type  are 
required  on  a  periodic  basis  (interval  determined  by  SOP)  and 
when  significant  changes  to  the  unit  status  occur.  The  causes 
of  these  changes  are  dependent  on  the  level  of  activity  in  the 
tactical  scenario.  Enemy  contact  will  result  in  losses  of 
combat  power,  expenditure  of  supplies,  and  changes  in  force 
disposition,  all  of  which  require  updates  to  the  unit  status 
in  addition  to  those  required  by  SOP.  At  high  levels  of 
OPTEMPO,  we  would  expect  these  tasks  to  arrive  in  a  manner 
similar  to  dynamic  tasks,  as  constant  changes  in  unit  status 
generate  the  update  task  and  few,  if  any  periodic  reports  are 
necessary.  At  low  levels  of  OPTEMPO,  we  observe  the  opposite 
with  very  few  changes  occurring  in  unit  status  between 
required  periodic  situation  reports. 
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Arrivals  of  periodic  tasks  at  low  levels  of  OPTEMPO 
should  occur  with  a  nearly  batch  process  behavior,  reflecting 
subordinate  units  submitting  situation  reports  at  or  about  a 
time  specified  by  SOP- -e.g. , the  SOP  might  dictate  subordinate 
units  to  submit  a  situation  report  on  the  hour,  plus  or  minus 
ten  minutes.  Outside  of  this  reporting  window  we  should 
observe  few  if  any  arrivals,  until  the  next  periodic  reports 
are  due  or  battlefield  activity  results  in  changes  to  unit 
status. 

Medium  levels  of  OPTEMPO  can  occur  when  only  part  of 
the  force  is  in  contact.  We  should  observe  periodic  reports 
arriving  from  units  in  reserve  or  in  quiet  sectors  and  tasks 
arriving  according  to  a  more  dynamic  process  from  units  in 
contact  (rapidly  changing  status) .  Modeling  periodic  task 
arrivals  presents  a  significant  challenge,  for  while  we  expect 
to  see  arrivals  one  at  a  time  and  at  a  rate  varying  with 
OPTEMPO,  we  must  also  account  for  variations  in  the  shape  of 
the  arrival  distribution  based  on  OPTEMPO. 

Interactive  tasks  are  those  with  task  generation 
dependent  solely  on  activities  within  the  C2  framework. 
Prepare  Fragmentary  Order  (PREP  FRAGO)  is  an  example  of  an 
interactive  task.  Frag  orders  are  prepared  in  response  to  a 
higher  headquarters  frag  order  or  as  a  result  of  critical 
events  in  the  tactical  scenario  (within  C2NET,  each  Process 
Critical  Sitrep  task  has  a  chance  of  generating  a  PREP  FRAGO 
task) .  Since  no  independent  arrival  process  is  required  for 
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interactive  tasks,  validating  the  model  behavior  of  dynamic 
and  periodic  tasks  will  satisfy  the  arrival  process  input 
distribution  validation  issue  for  C2NET. 

Service  times  in  the  real  system  do  not  appear  to 
depend  on  task  category  or  the  OPTEMPO  of  the  tactical 
scenario.  Once  a  task  arrives  at  a  server  (the  task  may  or 
may  not  have  waited  in  a  queue  for  a  server) ,  we  expect  the 
server  to  work  on  that  task  until  completed  (task  served  one 
at  a  time)  with  the  service  time  of  any  given  type  of  key  C2 
task  independent  of  previous  service  times  and  identically 
distributed.  For  example,  a  spot  report  of  a  certain  length 
and  complexity  may  require  one  minute  to  process  where  this 
processing  time  should  not  be  affected  by  the  presence  (or 
absence)  of  additional  spot  reports  in  the  queue,  nor  by  the 
time  to  process  previously  served  spot  reports. 

2.  Examination  of  Empirical  Data  Sources 


Sargent  (1980)  points  out  that  validation  should 
include  validation  of  the  data  used  to  develop  the 
conceptual  model,  and  the  data  used  for  testing.  It  is 
usually  time-consuming  and  costly  to  obtain  accurate  and 
appropriate  data,  and  initial  attempts  to  validate  a  model 
may  fail  if  insufficient  effort  has  been  applied  to  this 
problem.  A  model  that  merely  accommodates  known  facts  is 
just  an  ad  hoc  fit.  A  worthwhile  model  must  generate 
predictions  that  are  then  corroborated  by  observations  of, 
or  experiments  with,  the  real  system.  If  this  process 
leads  to  really  new  insight,  then  the  model  takes  on  the 
status  of  a  theory.  (Bratley,  Fox,  and  Schrage,  1987,  pp. 
8-9) 
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Before  selection  of  input  distributions,  it  is 
necessary  to  examine  what  data  will  be  used  to  develop  and 
test  our  model.  Although  much  has  been  done  by  Army 
analytical  agencies  in  collecting  observations  of  C2  systems 
in  action,  most  of  this  work  fails  Sargent's  criteria  as 
"accurate  and  appropriate  data"  when  applied  to  C2NET 
validation.  At  a  minimum,  empirical  data  to  support  our 
effort  should  be  current,  targeted  to  observation  of  key  tasks 
represented  in  C2NET,  and  with  a  sufficient  sample  size  to 
provide  a  clear  picture  of  potential  underlying  theoretical 
distributions  (also  insuring  statistical  significance  during 
eventual  hypothesis  testing) .  Data  collected  as  a  result  of 
two  different  studies  meet  these  criteria  and  are  used  in  our 
C2NET  validation  effort. 

The  first  data  source  is  from  a  1990  study  of  division 
level  C2  by  the  Army  Research  Institute  (ARI) .  Although  the 
collection  effort  was  for  ARI  internal  use  and  no  information 
was  gathered  on  task  processing  times,  the  data  regarding 
arrival  times  are  usable  for  analyzing  at  least  one  task.  Our 
primary  source  of  empirical  data  for  both  C2  task  arrival  and 
duration  is  from  a  C2  Analysis  Cell  study  of  SANTA  FE  V. 
SANTA  FE  V  was  a  brigade  level  command  post  exercise  (CPX) 
conducted  22-24  February,  1991  at  Fort  Leavenworth,  Kansas. 
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The  data  collected  from  these  studies  are  sufficient 
to  examine  the  following  tasks: 

•  Process  Intelligence  Spot  Report  (PIS,  Dynamic,  Division 
Level,  Arrival  Process  Only) 

•  Process  Critical  Situation  Report  (PCS,  Dynamic,  Brigade 
Level,  Task  Arrival  and  Duration  Data) 

•  Update  Status  of  Friendly  Forces  (UFF,  Periodic,  Brigade 
Level,  Task  Arrival  and  Duration  Data) 

The  small  number  of  tasks  represented  in  our  test  data 
(three  out  of  a  possible  105)  will  preclude  hypothesis  testing 
on  any  significant  scale  regarding  the  entire  database. 
However,  the  available  test  data  are  sufficient  to  allow 
examination  of  our  statements  about  the  behavior  of  C2  tasks 
through  comparison  of  empirical  distributions  with  the 
theoretical  distributions  suggested  from  our  knowledge  of  C2 
task  behavior.  Additionally,  hypothesis  testing  using  the 
empirical  data  will  generate  valuable  information  about  our 
conceptual  model- -e.g.,  p-values  and  confidence  intervals 
allow  us  to  draw  conclusions  about  model  resistance  to  errors 
in  parameter  estimation  and  quality  of  fit.  These  conclusions 
in  turn  will  indicate  potential  refinements  in  our  concepts  of 
the  behavior  of  C2  tasks,  with  subsequent  alterations  to  our 
model  resulting  in  predictions  consistent  with  Bratley's 
criteria  for  a  "worthwhile  model . " 
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3.  C2  Task  Arrival  Models 


One  question  that  may  arise  when  choosing  probability 
distributions  for  the  model  is  whether  to  use  frequency 
distributions  of  historical  data  or  to  seek  the 
theoretical  probability  distribution  which  best  fits  these 
data.  The  latter  alternative  usually  is  preferable 
because  it  would  seem  to  come  closer  to  predicting 
expected  future  performance  rather  than  reproducing  the 
idiosyncracies  of  a  certain  period  of  the  past.  (Law  and 
Kelton,  1991,  pp.  647-8) 


Given  our  previous  examination  of  the  behavior  of  the 
C2  task  arrival  processes  and  usable  test  data,  we  are  now 
prepared  to  search  for  the  appropriate  theoretical 
distributions  as  suggested  by  Law  and  Kelton. 

a.  Dynamic  Task  Arrival  Distributions 

The  behavior  of  dynamic  tasks  suggests  that  if  we 
let  the  random  variable  X  represent  the  arrival  times  for  one 
of  these  tasks,  then  X  is  independently  and  exponentially  (but 
not  identically)  distributed  with  arrival  rates  determined  by 
the  task's  BOS  OPTEMPO  (arrival  rates  are  piecewise  constant 
given  OPTEMPO) .  This  type  of  random  variable  is  said  to  be 
generated  by  a  nonstationary  (or  nonhomogeneous )  Poisson 
process.  He  turn  again  to  Law  and  Kelton  for  an  explanation 
of  the  attributes  for  this  process: 


Let  X(t)  be  the  arrival  rate  of  customers  to  some 
system  at  time  t.  If  customers  arrive  at  the  system  in 
accordance  with  a  Poisson  process  with  rate  X,  then  X(t)-X 
for  all  t*0.  However,  for  many  real-world  systems,  X(t) 
is  actually  a  function  of  t.  For  exanple,  the  arrival 
rate  of  customers  to  a  fast -food  restaurant  will  be  larger 
during  the  noon  rush  hour  than  in  the  middle  of  the 
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afternoon.  Also,  traffic  on  a  freeway  will  be  heavier 
during  the  morning  and  evening  rush  hours.  If  the  arrival 
rate  X{t)  does  in  fact  change  with  time,  then  the 
interarrival  times  A,,  A,, . . .  are  not  identically 
distributed;  thus,  it  is  not  appropriate  to  fit  a  single 
probability  distribution  to  the  A,'s... 

The  stochastic  process  (N(t),  t*0 }  is  said  to  be  a 
nonstationary  Poisson  process  if: 


1.  Customers  arrive  one  at  a  time. 

2.  N(t  +  e)  -  N(t)  is  independent  of  {N(u)  ,0su*t) . 


Thus,  for  a  nonstationary  Poisson  process,  customers 
must  still  arrive  one  at  a  time,  and  the  numbers  of 
arrivals  in  disjoint  intervals  are  independent,  but  now 
the  arrival  rate  X(t)  is  allowed  to  be  a  function  of  time. 
(Law  and  Kelton,  1991,  pp.  406-7) 


The  arrival  rate  as  a  function  of  time  is  the  key 
to  linking  the  force  evaluation  model's  tactical  scenario  with 
C2NET.  The  OPTEMPO  of  each  BOS  is  driven  by  the  tactical 
scenario.  Quantifying  this  OPTEMPO  by  BOS  over  the  life  of 
the  scenario  into  "OPTEMPO  Timelines"  and  inputting  the 
timelines  into  C2NET  as  governors  over  task  arrival  rates  will 
satisfy  our  linkage  requirement. 


The  importance  of  the  nonhomogeneous  Poisson  process 
resides  in  the  fact  that  we  no  longer  require  the 
condition  of  stationary  increments.  Thus  we  now  allow  for 
the  possibility  that  events  may  be  more  likely  to  occur 
during  certain  times  during  the  day  than  during  other 
times.  (Ross,  1989,  pg.  235) 
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The  approximation  made  in  categorizing  the 
continuous  distribution  of  OPTEMPO  into  three  discrete  levels 
as  High,  Medium,  and  Low  is  made  on  the  following  basis: 


•  This  is  a  sufficient  level  of  detail.  Examination  of  the 
test  data  indicates  three  levels  of  OPTEMPO  will  avoid 
significant  disparity  between  our  approximation  and  the 
real  system. 

•  This  is  a  practical  level  of  detail.  OPTEMPO  timelines 
must  be  created  for  each  Brigade  and  higher  unit,  for  each 
battlefield  operating  system,  for  every  scenario  C2NET  is 
to  approximate.  Additional  levels  of  detail  would 
increase  the  classification  work  significantly. 

•  This  is  a  useful  level  of  detail.  The  C2NET  database 
consists  of  three  estimates  for  MIN/MODE/MAX  interarrival 
times.  Our  hypochesis  for  task  arrival  processes  will 
link  this  database  with  parameter  estimates  at  the  three 
levels  of  OPTEMPO. 


The  first  step  in  analyzing  our  test  data  for 
conformity  with  the  nonstationary  Poisson  process  is  to  create 
the  OPTEMPO  timelines  for  the  applicable  BOS.  For  the  SANTA 
FE  test  data,  we  need  to  construct  Intelligence  and  Electronic 
Warfare  (IEW)  and  Maneuver  (MVR)  BOS  timelines  (for  the  PCS 
and  UFF  tasks  respectively) .  The  ARI  data  require  an  OPTEMPO 
timeline  for  the  IEW  BOS  (the  PIS  task) .  In  developing  the 
timelines  for  these  BOS,  no  information  was  available 
regarding  events  within  the  scenarios  and  the  timelines  in 
Figures  4  and  5  reflect  subjective  estimates  of  OPTEMPO.2 


2  A  methodology  is  discussed  in  Chapter  IV  as  a  guide  in 
developing  OPTEMPO  timelines  from  the  force  evaluation  model 
scenario. 
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These  estimates  are  based  on  the  test  data  and  background 
information  on  the  studies.  The  timeline  estimation  procedure 
may  have  introduced  error  into  our  study  through  bias  and 
incorrect  identification  of  timeline  OPTEMPO  levels.  Atteirpts 
to  reduce  error  through  independent  analysis  of  the  OPTEMPO 
timeline  will  not  totally  eliminate  these  types  of  error  and 
we  must  remain  aware  of  this  problem  during  our  examination  of 
the  input  process  as  a  potential  source  of  disparity. 

END  DAY  1  IND  DAY  2 
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Figure  4.  SANTA  FB  Study  OPTEMPO  Timelines 
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The  next  step  suggested  by  our  analysis  of  the 
arrival  behavior  of  dynamic  tasks  is  to  directly  compare  the 
test  data  against  the  exponential  distribution.  Using  the 
GRAFSTAT  analytical  tool,  we  will  jointly  plot  the  histogram 
of  the  test  data  interarrival  times  with  the  exponential 
density  function.  For  these  comparisons,  we  allow  GRAFSTAT  to 
select  the  input  parameter  yielding  the  best  exponential  fit 
to  our  test  data.  Figure  6  shows  the  plots  for  the  PCS  task 
from  the  SANTA  FE  study.  Plots  for  the  PIS  task  are  in  Figure 
7.  Additional  types  of  plots  (CDF  and  Probability- 
Probability  [P-P]  plots)  are  at  Appendix  B  comparing  test  data 
with  the  parameter  free  exponential  distribution. 


EXPONENTIAL  DENSITY  FUNCTION.  N-67  EXPONENTIAL  DENSITY  FUNCTION.  N-70 


Figure  6.  PCS  Histogram  with  Exponential  Distribution 
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Visual  inspection  of  these  plots  tends  to  confirm 
our  analysis  of  the  arrival  process  for  the  PCS  dynamic  task. 
Both  at  medium  and  high  OPTEMPO,  the  test  data  appear 
reasonably  well  fit  by  the  exponential  distribution.  At  this 
stage,  this  is  sufficient  to  continue  our  hypothesis 
development  process. 


anmaw.  Doarrr  m nctok.  *-ie 


UKHomt.  Bonrrr  nm mu. 


DWNomt  boishy  motok.  *»> 


Figure  7.  PIS  Histogram  with  Exponential  Distribution 

As  we  observed  with  the  test  data  for  the  PCS 
task,  examination  of  the  plots  for  the  PCS  dynamic  task  agrees 
with  the  suggestion  that  the  dynamic  task  arrival  process  is 
nonstationary  Poisson.  Each  plot,  from  high  to  low  OPTEMPO, 
appears  to  support  the  argument  of  an  underlying  exponential 
distribution,  with  rate  varying  by  OPTEMPO. 
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The  results  of  the  comparisons  made  using  test 
data  on  dynamic  tasks  against  the  exponential  distribution  are 
sufficiently  encouraging  to  continue  with  formal  hypothesis 
construction  and  testing.  First  however  we  must  complete  the 
selection  of  input  distributions  for  arrival  models  (periodic 
tasks)  and  task  duration  models. 

2>.  Periodic  Task  Arrival  Distributions 

Unlike  the  behavior  of  dynamic  tasks,  where 
arrivals  were  uniquely  identified  with  the  nonstationary 
Poisson  process,  no  ready  explanation  exists  for  the  arrival 
process  of  periodic  tasks.  We  do  know  from  the  behavior  of 
arrivals  that  the  exponential  should  be  the  underlying 
theoretical  distribution  for  periodic  tasks  during  periods  of 
high  OPTEMPO.  In  selecting  the  underlying  distributions  for 
periods  of  medium  and  low  OPTEMPO,  we  make  the  initial 
assumption  that  these  distributions  are  of  the  same  type,  with 
changes  in  shape  an*3  scale  to  approximate  the  increasingly 
periodic  nature  of  these  tasks.3  There  are  two  obvious 
candidate  distributions  that  include  the  exponential  (given 
the  correct  shape  parameter,  a«l) .  These  distributions  are 
the  Gamma  and  the  Weibull.  With  this  choice,  we  will 


3  In  C2NET,  we  will  not  attempt  to  simulate  the  actual 
periodic  nature  of  these  tasks,  only  the  workload  generated 
over  time.  The  resolution  of  interface  between  C2NET  and  the 
force  evaluation  model  does  not  justify  the  degree  of  detail 
required  to  deliberately  simulate  the  periodic  lulls  and 
peaks.  In  effect,  we  are  modeling  the  steady- state  arrival 
behavior  for  periodic  tasks  in  a  terminating  model. 
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initially  select  the  Gamma  distribution  for  analysis  with  the 
option  of  returning  to  the  Weibull  if  the  Gamma  proves 
unsatisfactory.  This  selection  is  based  on  robustness- -the 
Gamma  distribution  is  more  resistant  to  error  in  the  shape 
parameter. 

From  our  analysis  of  the  behavior  of  the  nature  of 
the  arrival  process  for  periodic  tasks,  we  know  to  set  a->l 
during  periods  of  high  OPTEMPO.  We  need  to  estimate  the  shape 
parameters  for  periods  of  medium  and  low  OPTEMPO.  Previous 
analysis  suggests  that  the  shape  parameter  for  medium  OPTEMPO 
should  be  less  than  that  for  low  OPTEMPO.  Initially,  we  will 
estimate  a*3  at  periods  of  low  OPTEMPO  and  a-2  for  medium 
OPTEMPO  while  allowing  the  scale  parameter  to  be  free 
(GRAFSTAT  selecting  0  to  best  fit  the  test  data)  .  The  fits 
obtained  proved  unsatisfactory  with  poor  behavior  in  the  tails 
and  local  modes  of  the  test  data  (see  Figures  23-26  in 
Appendix  B)  .  At  the  next  estimated  values  of  a-2  for  low 
OPTEMPO  and  a-1.5  for  medium  OPTEMPO  satisfactory  fits  were 
obtained.  Graphic  results  of  fitting  the  UFF  task 
interarrival  time  test  data  (partitioned  according  to  the 
maneuver  timeline  in  Figure  4)  against  the  Gamma  distribution 
(with  a-1/1.5/2  for  HI/MID/LOW  OPTEMPO  and  free  scale 
parameters)  are  shown  in  Figure  8. 
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Figure  8.  UFF  Histogram  with  Gamma  Distribution 

Visual  examination  of  the  plots  reveals  a  fairly 
good  fit  of  the  test  data  by  the  Gamma  distribution  with  shape 
parameter  estimates  as  discussed.  As  expected  for  the  medium 

and  low  levels  of  OPTEMPO  there  are  two  local  modes  formed  by 

« 

the  nature  of  periodic  reporting.  Selection  of  the  Gamma 
distribution  approximates  these  local  modes  into  a  singular 
mode,  which  is  satisfactory  given  our  interest  in  simulating 
workload  (see  Note  #3,  page  25).  It  may  be  of  interest  to 
future  analysts  to  accurately  model  the  presence  of  local 
modes  using  a  composite  or  piecewise  function.  However,  for 
purposes  of  this  paper,  the  Gamma  distribution  with  shape 
based  on  OPTEMPO  appears  to  sufficiently  approximate  the 
arrival  process  for  periodic  tasks. 
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4.  C2  Task  Duration  Modal 

In  C2NET,  each  task  performed  by  the  system  requires 
a  number  of  personnel  over  a  completion  period.  The 
completion  period  (or  duration)  is  determined  by  a  triangular 
distribution  with  parameters  estimated  by  subject  matter 
experts.  As  for  the  arrival  models,  identification  of  the 
underlying  theoretical  distribution  for  task  duration  will 
reduce  the  disparity  between  simulation  and  the  real  world 
behavior.  The  nature  of  C2  task  service  times  suggests  that 
we  examine  the  exponential  as  the  underlying  theoretical 
distribution  for  task  duration. 

To  test  the  exponential  distribution  we  have 
observations  made  of  the  PCS  and  UFF  tasks  during  the  SANTA  FE 
CPX.  There  are  two  problems  related  to  the  data  collection 
methodology  that  will  inpact  on  our  analysis.  The  first 
problem  is  with  the  discrete  measurement  of  duration.  The 
time  required  to  complete  a  task  in  this  study  was  measured  in 
minutes.  While  this  is  not  an  obvious  problem  in  our 
histogram  comparisons  (in  comparing  CDF  and  P-P  plots)  as  well 
as  computing  test  statistics,  the  discrete  measure  of  our  datr 
will  significantly  impair  the  results.  The  second  problem 
involves  time  spent  in  queues.  The  observations  made  for  task 
duration  are  based  solely  upon  the  time  a  task  arrived  and 
when  subsequently  completed.  No  information  is  available  on 
the  amount  of  time  a  particular  arrival  may  have  spent  in  a 
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queue  awaiting  processing.  With  these  caveats,  we  plot  the 
histograms  of  the  PCS  and  UFF  task:  duration  data  with  the 
parameter  free  exponential  distribution  at  Figure  9  and  Figure 
10,  respectively.  Cell  width  in  these  histograms  of  the  test 
data  is  fixed  at  l.l  to  insure  the  discrete  data  is  accurately 
represented. 


EXPONENTIAL  DENSriY  FUNCTION,  N-130 


Figure  9.  PCS  Duration  with  Exponential  Distribution 
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For  the  PCS  duration  data,  the  exponential 
distribution  appears  to  produce  a  satisfactory  fit,  confirming 
our  estimate  of  the  behavior  of  task  duration.  Conducting  a 
X1  goodness -of -fit  test  on  this  comparison  yields  a 
disappointing  p- value  of  only  .053,  highlighting  the  effects 
of  data  idiosyncrasies  on  eve’-  ;ual  hypothesis  testing.  For 
example,  the  absence  of  any  seven  minute  task  durations  is  a 
significant  contributor  to  the  high  x2  statistic. 


EXPONENTIAL  DENSITY  FUNCTION,  N=79 


Figure  10.  UFF  Duration  with  Exponential  Distribution 
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The  UFF  comparison  also  appears  to  support  the 
selection  of  the  exponential  as  our  underlying  theoretical 
distribution.  A  x2  test  gives  us  an  acceptable  p- value  of 
.253. 4 

Despite  the  suspect  results  obtained  with  the  PCS  task 
data,  we  can  continue  our  examination  of  duration  models  with 
some  confidence  the  underlying  theoretical  distribution  is  in 
fact  exponential. 

C.  INPUT  DISTRIBUTION  PARAMETER  VALUES 

The  next  step  in  developing  our  input  distributions  is  to 
identify  a  methodology  to  select  input  parameters.  From  our 
examination  of  OPTEMPO,  we  will  need  three  input  parameters 
for  each  task  representing  arrival  rates  at  HIGH/MID/LOW 
OPTEMPO.  We  must  also  determine  the  processing  rates  for  task 
durations.  The  obvious  suggestion  is  to  use  the  database 
values  for  MIN/MODE/MAX  interarrival  times  to  determine  scale 
parameters  in  dynamic  and  periodic  tasks'  arrival  processes. 
This  idea  is  not  without  justification,  since  the  database 
estimates  for  interarrival  times  were  initially  obtained  as 
expected  MIN/MODE/MAX  frequencies  of  arrivals.  Conversion  to 
MIN/MODE/MAX  estimates  for  interarrival  times  is  not  strictly 
accurate.  For  example,  if  the  estimate  for  expected  maximum 


4  We  cannot  reject  the  hypothesis  that  UFF  duration  data 
is  from  an  exponential  distribution  at  any  reasonable  critical 
value . 
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frequency  for  a  particular  task  in  a  one  hour  period  is  six, 
it  does  not  necessarily  follow  that  the  minimum  interval  for 
arrivals  is  ten  minutes.  To  obtain  the  subject  matter  expert 
estimate  for  maximum  arrivals  we  expect  an  average 
interarrival  time  of  ten  minutes.  The  database  values  are 
therefore  valid  estimates  for  calculating3  the  scale  of  the 
appropriate  dynamic  or  periodic  arrival  distribution. 

Formally  stating  our  hypotheses  concerning  the  use  of  the 
database  in  conjunction  with  our  previous  analysis  of 
underlying  distributions  and  the  nature  of  C2  systems  yields 
the  following. 

Bynarai-c  -Task  J&EiYalfi 

Ho.  Let  f  ( t)  represent  the  distribution  of 

interarrival  times  for  any  given  dynamic  task  at  time  t,  where 
0,  represents  the  point  estimate  of  the  mean  of  an  exponential 
distribution  at  time  t.  Then  we  can  approximate  f(t)  as  a 
composite  function  such  that: 

(EXP(0min)  »  t  e  High  OPTEMPO 
EXP(0mode)  #  t  €  Medium  OPTEMPO 
SXP(#max)  »  t  €  Low  OPTEMPO 


3  For  exponential  distributions,  our  scale  estimate  is 
simply  the  database  value.  For  Gamma  distributions,  the  mean 
is  given  by  a/3  so  with  the  database  estimate  of  the  mean 
(denoted  by  0mzn/mode/max)  >  we  calculate  $hi/mid/low“ 0 min/mode/max / ® • 
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Periodic.  Task  Arrivals 


Ho:  Let  g(t)  represent  the  distribution  of 


interarrival  times  for  any  given  periodic  task  at  time  t, 
where  6,  represents  the  point  estimate  of  the  mean  of  a  Gamma 
distribution  at  time  t.  Then  we  can  approximate  g(t)  as  a 
composite  function  such  that: 


g(t)  - 


GAMMA  (1, 6m) 

•  GAMMA(1.5, 0Mode/1-5) 
GAMMA  (2 , 0max/2  ) 


t  e  High  OPTEMPO 
t  €  Medium  OPTEMPO 
t  €  Low  OPTEMPO 


TasK  Duration 


Ho:  Let  h(t)  represent  the  distribution  of  completion 


times  for  any  given  task  (to  include  interactive  tasks)  at 
time  t,  where  6,  represents  the  estimate  of  the  mean6  of  an 
exponential  distribution  at  time  t.  Then  we  can  approximate 
h(t)  such  that: 


h(t)  -  -(EXP^mjn)  ,  V  t 

These  three  distributions  (£,  g,  and  2i)  incorporate  our 
knowledge  of  C2  task  behavior  and  underlying  distributions. 
Successful  testing  of  these  distributions  against  our  test 


6  Prom  our  analysis  of  the  nature  of  task  durations,  we 
should  need  only  one  point  estimate  for  the  mean.  Selection 
of  the  Minimum  expected  coittpletion  time  from  the  database  is 
founded  on  the  results  obtained  in  our  earlier  examination  of 
UFF  and  PCS  duration  data.  Future  data  collection  efforts 
should  be  conducted  in  this  area  to  insure  the  conclusions 
reached  in  this  paper  (e.g.,  MIN  value  from  database  as 
estimate  for  task  duration  mean )  are  accurate,  given  the 
previously  stipulated  problems  with  task  duration  test  data. 
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data  will  validate  in  principle  our  effort  to  refine  the  input 
process  of  C2NET,  with  the  benefit  of  linking  the  force 
evaluation  model's  scenario  into  the  activities  of  C2NET. 


D.  HYPOTHESES  TESTING  AND  INPUT  VALIDATION 

We  are  now  prepared  to  test  our  input  distributions  and 
associated  parameters  from  the  C2NET  database  against  our  test 
data.  Table  1  summarizes  the  data  values  appropriate  to  each 
test.  These  values  include  the  database  estimate  of  the  mean 
(8) ,  specified  shape  and  scale  parameters,  and  for  comparison 
purposes,  95%  confidence  intervals  for  the  scale  parameter 
identified  during  our  examination  of  the  parameter  free 
theoretical  distributions . 

TABLE  1.  DATA  VALUES  FOR  HYPOTHESES  TESTING7 


MEAN 

ESTIMATE 

SHAPE 

PARAMETER 

SCALE 

PARAMETER 

98% 

C.I. 

ARRIVAL  PROCESS 

■  i  i 

ISO 

NA 

15 

144213 

wzsxziMmm 

7.5 

NA 

7.5 

34108 

PIS  LOW 

60.0 

NA 

60 

309,705 

wunznamm 

ISO 

NA 

30* 

224483 

PIS  HIGH 

7.5 

NA 

7.6 

702,179 

UFFLOW 

00.0 

2 

16* 

UFF  MEDIUM 

ISO 

1.5 

10 

34143 

IFF  HIGH 

7.5 

1 

15* 

01,173 

SERVICE  PROCESS 

SO 

NA 

3 

132333 

|  UFF  (ALL  OPTEMPO) 

SO 

NA 

3 

£31446 

7  Our  hypothesis  fails  using  the  mean  estimates  from  the 
database  for  tasks  marked  with  in  the  Scale  column.  These 
failures  are  attributable  to  incorrect  SME  estimates,  and  new 
estimates  that  are  not  in  conflict  with  empirical  data  are 
used  as  shown. 
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Again  using  the  GRAFSTAT  analytical  tool  and  the  data  from 
Table  1,  we  compare  the  test  data  with  the  appropriate 
theoretical  distribution.  Figures  11-15  show  the  CDF  plots 
from  each  of  these  comparisons.  Results  of  x2  goodness- of - 
fit  tests  are  in  Table  2  for  arrival  processes  and  Table  3  for 
duration  processes.  Additional  plots  are  at  Appendix  B  (see 
Figures  18-22  for  arrival  processes  and  Figures  27-28  for  task 
durations) . 

EXPONENTIAL  CUMULATIVE  DISTRIBUTION  FUNCTION,  N-67  EXPONENTIAL  CUMULATIVE  DISTRIBUTION  FUNCTION.  N-7C 


Figure  11.  PCS  and  Exponential  Distribution  CDF  Plots 

At  high  OPTEMPO,  the  CDF  plot  shows  a  remarkably  good  fit. 
The  medium  OPTEMPO  plot  indicates  that  our  estimate  of  the 
mean  (0* 15)  is  above  the  true  mean  of  the  sample  population. 
The  generated  p-value  of  0.024  confirms  our  observation.  The 
probable  reason  for  this  disparity  is  the  presence  of  low 
OPTEMPO  data  in  our  test  sample,  lengthening  the  tail  of  the 
test  data. 
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Figure  12.  PIS  and  Exponential  Distribution  CDF  Plots 


All  of  these  comparisons  appear  well  behaved.  In  high  and 
low  OPTEMPO,  there  is  some  disparity,  but  in  general  the  fits 
support  the  hypotheses  regarding  the  f(t)  distribution. 


Figure  13.  UFF  and  Gamma  Distribution  CDF  Plot 


Using  the  corrected  values  for  estimates  of  the  mean 
result  in  the  solid  fits  as  shown  in  Figure  13.  The  right 
tail  at  high  and  low  OPTEMPO  are  longer  under  our.  hypothesis 
of  g(t)  than  in  the  test  data,  but  not  critically  so.  Note 
that  the  local  modes  so  apparent  in  the  histogram  plots  are 
transparent  in  the  CDF  plots  at  medium  and  low  OPTEMPO.  Test 
statistics  generated  from  our  examination  of  arrival  sample 
data  are  as  follows: 

TABLE  2.  ARRIVAL  HYPOTHESES  TEST  RESULT  SUMMARY 


1 

TEST  STATISTIC 

CM 

WKSZSnm 

w:±vh.'  i '■ 

9.40 

S 

0.024 

PCS  HIGH 

2.77 

5 

0.735 

PIS  LOW 

2.41 

3 

0.491 

PIS  MEDIUM 

1.01 

3 

0.797 

PISHK3H 

IRS 

2 

0.536 

UFF  LOW 

7.18 

S 

0.206 

ili 

2.67 

3 

0.445 

i  UFFHK3H 

2.04 

4 

0.728 

Similarly,  we  conduct  comparative  plotting  and  generate 
test  statistics  for  the  hypothesis  regarding  task  duration. 
Figures  14  and  15  depict  CDF  and  P-P  plots  for  PCS  and  UFF 
duration,  respectively. 

The  CDF  and  P-P  plots  of  the  PCS  duration  data  with  the 
exponential  distributions  highlight  the  problem  of  discrete 
data  collection  for  the  continuous  service  time  process.  The 
X2- statistic  generated  from  this  comparison  reflects  the  poor 
quality  of  the  data  with  a  p- value  of  0.00022.  The  poor  fit 
in  the  tails  is  the  primary  source  of  disparity. 
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EXPONENTIAL  PROMUTY  PLOT,  N-130 


MX  OPIDFO — OUMTION  MI  OfltWO — OWWTON 


Figure  14.  PCS  Duration  and  Exponential  Distribution  Plots 


EXPONENTIAL  CUMUUOWE  CXtTRBUTTON  FUNCTION,  N-71  EXPONENTIAL  PROMUTY  PUTT,  N-71 


ML  OPTOPO—OUNATON  MX  OPTEMTO — OUNOION 


Figure  15.  XJFF  Duration  and  Exponential  Distribution  Plots 

Although  we  observe  the  same  problem  with  the  discrete 
test  data  as  for  the  PCS  plots,  the  UFF  duration  data  appear 
to  be  well  fit  by  the  h{t)  distribution  and  supports  our  null 
hypothesis . 
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TABLE  3.  DURATION  HYPOTHESIS  TEST  RESULT  SUMMARY 


TEST  STATISTIC 

dt 

p-VALUE 

PCS  (ALL  OPTEMPO) 

26.03 

6 

0.00022 

UFF  (ALL  OPTEMPO) 

7.06 

6 

0.16400 

The  p- values  generated  by  tests,  except  where  noted, 
support  the  hypothetical  distributions  f,  g,  and  h. 
Disparities  between  these  distributions  and  the  test  data  are 
primarily  attributable  to  errors  in  the  data  collection 
methodology  (affecting  primarily  the  duration  tests) ,  errors 
in  the  database  estimates  (UFF  at  low  and  high  OPTEMPO  and  PIS 
at  medium  OPTEMPO)  ,  and  idiosyncracies  in  the  test  data.  With 
the  correct  estimates  of  mean  and  accounting  for  the  other 
sources  of  error,  the  test  results  strongly  support  the 
validity  of  our  procedure  for  modeling  C2  task  input  processes 
in  C2NET. 

Our  analyses  have  thus  far  centered  on  improving  the  C2NET 
model  through  enhancing  the  input  process.  Our  next  concern 
must  be  to  examine  the  C2NET  output  and  how  it  is  used  in  the 
force  evaluation  model.  In  the  next  chapter  we  will  conduct 
this  examination,  again  with  the  intent  of  identifying 
disparities  and  analyzing  potential  solutions  to  reduce  the 
impact  on  model  performance. 


39 


III.  C2NET  OUTPUT  VALIDATION 


In  actual  usage,  of  course,  models  are  typically 
applied  to  situations  not  strictly  represented.  This  may 
be  done  deliberately,  and  one  then  speaks  of  ingeniously 
"gimmicking"  the  model.  If,  for  example,  a  land  warfare 
model  does  not  strictly  represent  supporting  air  power, 
but  does  represent  artillery,  then  in  application  air 
power  may  be  represented  as  artillery.  Strictly  speaking, 
however,  any  validation  of  such  a  model  in  such  an 
extended  application  should  include  validation  of  the 
"rules"  of  extension.  How,  that  is,  does  one  represent 
air  power  as  artillery?  How  are  the  numerical  values  of 
the  inputs  to  be  selected?  (Thomas,  1989,  pg.  265) 


Validation  of  C2NET  prompts  us  to  ask;  how  does  one 
represent  command  and  control  in  a  force  evaluation  model? 
Specifically,  how  do  we  use  the  output  from  C2NET  as  input 
into  VIC  to  simulate  C2  effects  on  Division/Corps  operations? 
This  is  a  significant  challenge  given  the  deterministic  design 
of  VIC  and  the  imprecise  nature  of  C2.  The  first  step  in  the 
validation  of  C2NET  output  is  analysis  of  the  "rules  of 
extension"  currently  in  use  to  approximate  C2  in  VIC.  We  will 
then  examine  a  new  method  for  portraying  C2  effects  through 
application  of  fuzzy  set  theory.  Finally,  we  will  construct 
and  discuss  new  rules  of  extension  using  our  fuzzy  methodology 
to  apply  C2NET  output  within  VIC. 
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A.  ANALYSIS  OF  CURRENT  C2NET  OUTPUT  PARAMETERS 

C2NET  is  designed  to  measure  the  responsiveness  of  C2 
systems  with  VIC  used  to  determine  the  impact  of  this 
responsiveness  on  force  effectiveness.  Currently,  C2  effects 
input  to  VIC  consist  entirely  of  delay  times  influencing 
execution  of  strike  missions1  and  maneuver  orders  as  output 
from  C2NET.  These  delay  inputs  are  summarized  as  follows: 


C2NET  outputs  (queuing  and  processing  delays)  are 
inputs  to  VIC.  Three  types  of  delays  will  be  adjusted  in 
VIC- -  fusion,  decision,  and  execution.  Fusion  delays  may 
be  entered  for  each  sensor  --  without  regard  to  the 
echelon  supported.  Decision  delays  are  of  two  types  -- 
artillery  and  maneuver.  VIC  will  accept  artillery 
decision  delays  at  each  echelon  --  corps,  division,  and 
brigade.  A  single  maneuver  decision  delay  may  be  entered 
at  each  dwell  point  along  a  maneuver  unit's  path  --  corps, 
division,  and  brigade.  Execution  delays  may  be  entered 
for  artillery  by  attack  means.  (Ramaden,  1991,  pg.  7-1) 


The  delays  represent  the  following: 


•  Fusion  Center  delays  in  processing  sensor  reports  into 
strike  target  nominations. 

•  Decision  delays  in  the  staff  process  for  processing  target 
nominations  and  movement  orders. 

•  Execution  delays  from  receipt  of  strike  mission  order  to 
rounds  on  the  way. 


1  "Strike"  missions  include  tube  and  missile  artillery 
fires,  attack  helicopters  (AH) ,  and/or  USAF  close  air 
support/battlefield  interdiction  (CAS/BI)  sorties. 
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The  fusion  and  execution  delay  factor  inputs  into  VIC  are 
estimates  from  the  appropriate  service  schools.  Decision 
delays  are  calculated  from  appropriate  process  delays  observed 
in  C2NET  runs.  When  combined,  these  delay  factors  determine 
the  maneuver  and  request  for  strike  mission  asset  delay 
factors  to  be  entered  into  VIC. 

Two  problems  are  associated  with  the  current  method  by 
which  C2  responsiveness  is  approximated  in  VIC.  The  first, 
most  obvious  problem  is  the  aggregation  of  delays  into  single 
point  estimates  by  echelon  for  maneuver  forces  and  delivery 
means  for  strike  assets.  As  discussed  previously  in  our 
analysis  of  input  processes,  we  expect  the  C2  system  workload 
to  vary  with  tactical  intensity  producing  variations  in  system 
responsiveness.  The  second  problem  was  summarized  by  Doctor 
La  Rocque,  Director  of  the  TRADOC  Operations  Analysis  Center 
at  Fort  Leavenworth,  "Command  and  Control  is  not  just  delay 
times."  While  the  aggregation  of  times  required  to  complete 
key  C2  tasks  is  a  valid  approximation  of  decision  delay, 
delays  alone  are  insufficient  approximations  of  C2  effects  in 
a  force  evaluation  model.  In  particular,  VIC  as  a 
deterministic  model  with  scripted  maneuver  fails  to  simulate 
the  full  effect  maneuver  and  strike  mission  delays  have  when 
combined  with  all  other  mission  attributes. 
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To  demonstrate,  we  will  examine  the  concept  of  artillery 
effectiveness.  Any  given  artillery  mission's  effectiveness  is 
dependent  on  a  number  of  variables  as  represented  in  Figure 
16.  The  decision  process,  isolated  from  the  other  explanatory 
variables,  fails  to  accurately  simulate  the  compound  effects. 
For  example,  a  ten  minute  delay  between  call  for  fire  and 
rounds  on  the  way  may  be  satisfactory  for  a  slow  moving  target 
in  open  terrain  where  observation  may  be  maintained  and  fires 
accurately  adjusted.  However,  if  the  target  is  moving  quickly 
or  in  broken  terrain,  this  delay  may  cause  the  fires  to  be 
wholly  ineffective.  VIC  in  its  current  form  does  not  account 
for  these  additional  factors  given  a  delay  value;  it  simply 
delays  the  application  of  attrition  for  the  required  time. 
Similar  problems  are  encountered  with  maneuver  delays  scripted 
into  "dwell  points"  for  moving  forces. 


Figure  16.  Conceptual  Hierarchy- •Artillery  Effectiveness 


43 


B.  EXAMINATION  OP  SOLUTIONS  TO  OUTPUT  DISPARITIES 

Our  previous  analysis  of  the  C2NET  input  process  will 
serve  as  the  basis  for  solving  the  aggregation  problem.  The 
current  input  process  relies  on  single  input  distributions  for 
each  task  by  echelon.  Thus  the  resolution  of  the  output  is 
reasonable  given  the  aggregation  of  the  input.  By  increasing 
the  level  of  detail  in  C2NET  to  each  unit  over  time 
(represented  by  OPTEMPO  timelines  for  each  unit  over  the  life 
of  the  scenario)  as  proposed,  we  gain  the  additional  benefit 
of  output  for  each  unit  over  time.  This  segregation  will 
improve  our  approximation  of  the  effects  of  C2  responsiveness 
within  VIC  significantly,  as  decision  delay  factors  are  now 
linked  to  the  tactical  scenario  for  each  unit  and  not  simple 
point  estimates  by  echelon  or  attack  means  for  the  entire 
scenario. 

Accurately  capturing  the  effects  of  C2  responsiveness  in 
our  force  evaluation  model,  beyond  delay  times,  is  resistant 
to  a  simple  solution.  As  in  our  artillery  effectiveness 
example,  delays  in  the  decision  and  execution  of  a  fire 
mission  will  certainly  have  an  effect  on  its  outcome.  Our 
imprecise  knowledge  of  the  cumulative  effect  of  responsiveness 
with  other  explanatory  variables  suggests  a  potential 
methodology  to  define  C2  effects  in  a  force  evaluation  model- - 
fuzzy  set  theory. 


sagr* 


1.  Basics  of  Pussy  Sat  Theory 


In  classical  normative  decision  theory  the  components 
of  the  basic  model  of  decision  making  under  certainty  are 
taken  to  be  crisp  sets  or  functions.  By  crisp  we  mean 
dichotomous- -that  is,  of  the  yes-or-no  type  rather  than  of 
the  more -or- less  type.  The  set  of  actions  is  as  precisely 
defined  as  the  set  of  possible  states  (or  the  state)  and 
the  utility  function  is  also  assumed  to  be  precise. 

Fuzzy  set  theory  provides  a  strict  mathematical 
framework. .. in  which  vague  conceptual  phenomena  can  be 
precisely  and  rigorously  studied.  It  can  also  be 
considered  as  a  modelling  language  well  suited  for 
situations  in  which  fuzzy  relations,  criteria,  and 
phenomena  exist.  (Zimmermann,  1987,  pp.  10-11) 


Building  upon  this  difference  between  classical  and 
fuzzy  sets,  Zimmermann  formally  defines  a  fuzzy  set  as 
follows : 


If  X  is  a  collection  of  objects  denoted  generically  by 
x  then  a  fuzzy  set  A  in  X  is  a  set  of  ordered  pairs: 

&  ■  {  (x,  fiA  (x) )  |  X€X} 

HA  is  the  membership  function  that  maps  X  to  the  member¬ 
ship  space  M  and  nA(x)  is  the  grade  of  membership  (also 
degree  of  compatibility  or  degree  of  truth)  of  x  in  A. 

Fuzzy  set  theory  appears  ideally  suited  to  assist  us 
in  defining  and  quantifying  the  imprecise  nature  of  C2  effects 
in  an  evaluation  model.  Applying  this  definition  to  our 
artillery  effectiveness  problem  will  give  us  a  clearer  picture 
of  the  utility  of  fuzzy  set  theory  in  the  C2  responsiveness 
problem.  For  example,  we  want  to  classify  the  performance  of 
artillery  in  direct  support  of  maneuver  units.  One  indicator 
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of  the  effectiveness  of  artillery  is  the  delay  time  between 
request  for  fire  and  rounds  on  the  way.  Let  3, . . . , 10} 
be  the  set  of  possible  delay  times  (in  minutes)  .  Then  the 
fuzzy  set  "effective  artillery  fire  against  a  stationary 
target"  may  be  described  as 

X-{(1,1) ,  (2, .98) , (3, .95) ,  (4, .9) ,  (5, .8) , 

(6, .65) ,  (7, .5) ,  (8, .25) ,  (9, .1) .  (10,0.0) } 

In  this  example,  a  one  minute  delay  is  always  mapped 
into  the  fuzzy  set  "effective  artillery  fire,"  while  a  seven 
minute  delay  has  a  0.5  degree  of  membership  to  this  set.  Note 
that  the  membership  function  in  this  example  is  monotone. 
This  is  not  a  requirement  and  most  such  functions  are  not 
monotone  (it  is  therefore  inappropriate  to  consider  this 
function  as  some  type  of  cumulative  distribution  function  or 
attempt  to  construct  a  corresponding  probability  function) . 
To  completely  describe  "effective  artillery  fire  given  target 
speed"  in  this  example  we  would  also  be  interested  in 
constructing  the  fuzzy  sets  for  slow,  moderate,  and  fast 
moving  ground  targets.  This  allows  us  to  generalize  artillery 
effectiveness  for  all  target  types;  whether  an  enemy  defense 
or  threat  tank  company  halted  in  a  minefield,  an  infantry 
platoon  advancing  under  small  arms  fire  (slow) ,  a  motorized 
rifle  battalion  attacking  cross-country  (moderate)  ,  or  a  truck 
convoy  on  a  highway  (fast) . 


2.  Application  of  Fuzzy  Set  Theory  in  C2 

Before  proposing  a  solution  to  the  C2  responsiveness 
issue  utilizing  fuzzy  set  theory,  we  need  to  refine  our  model. 
The  fuzzy  sets  as  presented  in  our  model  consider  only  one 
additional  explanatory  variable  (target  speed) .  We  must  also 
keep  in  mind  that  the  delay  times  provided  from  C2NET  are  only 
approximations  of  average  system  performance  at  any  given  time 
in  the  battle.  Given  the  deterministic  nature  of  VIC,  this  is 
quite  satisfactory  (our  alternative  is  to  randomly  sample  some 
sort  of  empirical  distribution  produced  by  C2NET  for  delay 
times- -unacceptable  in  the  VIC  environment). 

To  account  for  the  contributions  of  other  (albeit  less 
significant)  variables  and  the  approximations  made  in  delay 
factors,  we  will  introduce  the  additional  concept  of  "fuzzy 
restrictions."  In  our  artillery  example,  these  restrictions 
are  identified  as  the  following:  Ineffective,  Poor,  Fair, 
Good,  and  Excellent  Effectiveness.  These  restrictions  will 
serve  to  aggregate  the  membership  functions  of  the  fuzzy  sets 
to  a  reasonable  level  of  resolution  given  our  imprecise 
knowledge  about  the  system.  We  can  also  establish  penalties 
to  the  effectiveness  (attrition  coefficients)  of  artillery 
missions  associated  with  each  value  of  effectiveness,  with 
eventual  application  to  portraying  C2  effects  in  the 
evaluation  model.  In  this  extension  of  the  .artillery  problem, 
any  given  delay  time  may  be  a  member  of  one  or  more 
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restriction  variables  (e.g.f  a  certain  delay  time  that  has  a 
0.5  degree  of  membership  to  the  " Excellent "  category  is  at  the 
same  time  almost  certainly  in  the  "Good"  category) .  The  next 
step  is  to  determine  for  each  delay  time  (x)  the  corresponding 
degree  of  membership  (fiA(x))  into  each  fuzzy  restriction 
(where  membership  values  will  differ  from  the  fuzzy  set 
examined  previously)  .  At  Figure  17,  there  is  an  example  of 
how  delay  time  memberships  are  mapped  into  the  fuzzy 
restrictions  Excellent  through  Poor. 


Figure  17.  Artillery  Effects  with  Fuzzy  Restrictions 

Analyzing  the  fuzzy  restriction  "Good  Artillery 
Effectiveness,"  for  example,  indicates  that  a  delay  of  five 
minutes  will  always  be  a  member  of  this  set.  At  ten  minutes 
there  is  a  0.7  degree  of  membership  that  this  delay  will 
result  in  good  effectiveness,  while  at  15  minutes  the 
corresponding  mapping  to  a  Good  result  is  down  to  0.2. 
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The  final  step  in  developing  a  satisfactory  method  for 
modeling  C2  responsiveness  in  a  force  evaluation  model  is  to 
design  a  nonstochastic  function  mapping  key  variables  into  the 
fuzzy  restrictions.  The  recommended  method  requires  we  depart 
from  the  fuzzy  construct  for  mapping  the  effectiveness 
function  in  VIC.  Doctor  Pat  Allan  of  the  Rand  Corporation 
suggested  this  method  in  a  phone  conversation  with  the  author 
(19  August,  1992)  ,  as  an  adaptation  of  the  Rand  "Able  Tables." 
This  method  is  based  on  our  existing  model  consisting  of  a 
fuzzy  variable  ("Artillery  Effectiveness")  with  restrictions 
("Good,"  "Poor,"  ...)  and  corresponding  mappings  of  the  degree 
of  membership  for  each  base  variable  into  these  restrictions 
(e.g.,  a  five  minute  delay  has  a  degree  of  membership  of  0.8 
in  the  set  of  "Excellent  Artillery  Effectiveness"  and  1.0  to 
the  set  "Good  Artillery  Effectiveness").  Using  this  model, 
the  analyst  must  then  set  a  minimum  degree  of  membership 
constraint  (or  a-level) .  The  a-level  limits  the  possible 
restrictions  for  mapping;  the  degree  of  membership  must  equal 
or  exceed  this  level  to  be  mapped  into  a  restriction.  Given 
the  set  of  possible  mappings  after  application  of  the  a-level, 
the  delay  times  are  subsequently  mapped  into  the  best  possible 
restriction.  In  the  artillery  example,  if  we  set  our  a-level 
at  0.5,  then  a  five  minute  delay  is  mapped  into  "Excellent," 
and  a  ten  minute  delay  into  "Good"  (since  the  0.3  degree  of 
membership  into  "Excellent"  is  less  than  our  a-level  and  the 
"Good"  variable  is  preferable  to  "Fair")  .  Thus  for  each  delay 
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time  and  associated  explanatory  variable (s)  in  VIC,  there  is 
only  one  corresponding  effectiveness  value  satisfying  our 
requirement  for  constructing  a  nonstochastic  function  mapping 
key  variables  into  fuzzy  restrictions. 

Continuing  with  the  artillery  example,  in  Table  4  we 
identify  the  value  of  artillery  mission  effectiveness  given 
the  explanatory  variable  "target  speed"  (known  to  VIC  for  each 
mission)  and  the  current  total  delay  time  from  request  to  fire 
(from  C2NET  output) .  The  responses  in  the  table  range  from 
Excellent  (E)  to  Ineffective  (I) .  The  numbers  in  parentheses 
indicate  the  portion  of  the  original  attrition  coefficient 
applied  under  a  given  response- -e.g. ,  a  Good  mission  will 
apply  90%  of  its  computed  attrition  coefficient.  This  type  of 
response  matrix  is  easily  input  into  VIC  and  will  not  require 
any  significant  restructuring  of  existing  VIC  decision  tables. 
The  largest  investment  involved  in  implementing  this 
methodology  is  in  performing  the  sensitivity  analysis  required 
to  set  penalty  values  at  the  appropriate  levels. 
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TABLE  4.  ARTILLERY  EFFECTIVENESS  MATRIX 


XELAH 

(MIN) 

STATIONARY 
(0  km/hr) 

SLOW 
(<  10  Km/hr) 

MODERATE 
(<  25  km/hr) 

FAST 

(>  25  km/N) 

1 

E  (IX) 

E(1X| 

E(1J0) 

GfOJ) 

2 

E(1X| 

EflXI 

GfM) 

FM 

3 

E(U0) 

G<M) 

G(0J) 

FM> 

4 

E(U0) 

<3(&G 

GfDkfl) 

P(Ofi) 

5 

E(U0) 

FfOJfi) 

p  eo.fi) 

6 

FfO-S) 

FfO.fi) 

IfBJO) 

7 

FfW) 

P(Ofi) 

IfBJO) 

8 

QffUf 

FfBA) 

P(0Lfi) 

IfBJO) 

9 

P(Ofi) 

P(0J) 

IfBJO) 

10 

P(0Lfi) 

IfBJO) 

IfBJO) 

KEYi 

E- EXCELLENT 
G-GOOO 
F-FAIR 
P-POOR 
I -INEFFECTIVE 


NOTE:  The  values  in  parentheses  reflect 
adjustments  to  a  fire  mission's  attrition 
coefRctent  when  it  has  the  given  effectiveness, 
(e  g.,  INEFFECTIVE  mtosions  have  no  effect) 


At  first  glance,  this  methodology  appears  to  entail  a 
significant  effort  to  produce  a  simple  response  table. 
However,  we  must  keep  in  mind  that  this  simple  table  is  an 
approximation  of  the  corresponding  fuzzy  set.  Only  through 
analysis  of  the  response  as  a  fuzzy  variable  are  we  able  to 
define,  quantify,  and  eventually  approximate  its  behavior 
accurately.  The  additional  benefit  of  this  methodology  is  its 
generality- -the  potential  for  solving  other  ill-defined 
problems . 

Suppose  we  are  interested  in  comparing  the  "Agility" 
of  friendly  forces  equipped  with  different  C2  hardware  in  an 
evaluation  model  using  C2NBT  output.  The  procedure  for  this 
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comparison  using  fuzzy  set  theory  would  follow  that  used  in 
our  artillery  example: 

•  Determine  contributing  key  C2  tasks 

•  Calculate  delays  using  C2NET 

•  Identify  significant  explanatory  variable (s)  from  the 
evaluation  model 

•  Construct  fuzzy  set(s)  with  restrictions  and  corresponding 
penalties’ 

•  Approximate  the  "Effectiveness  Matrix"  using  fuzzy  sets, 
restrictions,  and  resulting  penalties  over  the  range  of 
possible  delays  and  other  explanatory  variables 

•  Input  the  Matrix  and  run  evaluation  model  (may  require 
sensitivity  analysis  before  test  run) 

Our  new  model  of  delays  and  corresponding  penalties 
will  significantly  reduce  the  disparity  between  the  real 
system  and  how  we  approximate  the  effects  of  C2  in  the  force 
evaluation  model.  How  much  better  is  the  chance  of  a 
successful  air  strike  or  artillery  mission  against  an 
attacking  enemy  slowed  or  stopped  by  an  obstacle?  Likewise 
the  model  now  captures  the  difficulty  of  putting  effective 
fires  on  a  target  moving  unhindered  on  a  road.  Historically, 
most  fire  missions  shot  against  such  a  target  will  fall  on 
empty  ground.  Proper  tactics  are  rewarded;  construction  of 
engagement  areas  with  obstacles  (natural  and  emplaced)  will 


’  Penalties  can  be  designed  to  effect  attrition 
coefficients,  movement  routes/rates,  resupply,  ....  The 
analyst  must  determine  what  is  appropriate  and  workable  within 
the  evaluation  model  to  reflect  performance. 
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slow  and  constrict  enemy  maneuver  into  friendly  direct  and 
indirect  fires  and  in  deep  attacks,  interdiction  of  bridges 
and  road  junctions  will  stop  or  slow  reinforcing  columns  to 
enhance  artillery,  missile,  and  air  strike  conditioning  of 
these  forces. 

Similar  applications  of  fuzzy  set  theory  to  maneuver 
and  fusion  delay  are  possible  using  the  procedure  as  outlined 
previously.  Since  maneuver  is  scripted  in  VIC,  penalties 
associated  with  maneuver  effectiveness  will  probably  entail 
extension  of  the  original  scenario  to  include  additional  paths 
from  each  dwell  point,  allowing  units  to  select  the  path  taken 
based  on  ef f iciency- -e.g. ,  at  poor  effectiveness,  the  unit 
moves  to  indecisive  terrain. 

C.  THE  FINAL  MODEL 

The  proposed  changes  and  enhancements  to  C2FAM  are  all 
designed  to  increase  model  fidelity  and  decrease  disparity 
between  the  real  system  and  how  we  approximate  this  system. 
By  introducing  the  concept  of  operational  tempo  for  each  unit 
by  battlefield  operating  system,  we  can  quantify  the  tactical 
scenario  of  the  evaluation  model  into  a  form  usable  by  C2NET. 
Identifying  the  correct  underlying  theoretical  distribution 
for  each  key  C2  task  and  using  the  existing  subject  matter 
expert  estimates  to  define  shape  and  scale  parameters  under 
the  different  levels  of  operational  tempo  will  allow  us  to 
more  accurately  simulate  the  task  arrival  and  duration 
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processes.  We  gain  the  additional  benefit  of  higher 
resolution  output  by  segregating  C2  responsiveness  by  unit 
over  the  life  of  the  scenario.  Finally,  by  examining  the 
output  as  a  fuzzy  phenomena,  we  will  be  able  to  make 
nonstochastic  statements  about  system  effectiveness  (as  a 
result  of  the  C2  process)  within  the  evaluation  model.  These 
statements  with  corresponding  adjustments  to  the  behavior  of 
forces  will  enhance  the  quality  of  the  model's  performance  by 
better  capturing  the  effects  of  C2  responsiveness  on 
operational  capabilities. 

The  net  result  of  this  proposal,  if  implemented,  will  be 
a  model  truer  to  the  real  system  through  better  approximation, 
higher  resolution,  and  an  efficient  model  interface. 
Implementing  some  portions  of  this  proposal  will  be  simple 
(e.g.,  changing  the  distributions  for  arrival  and  task 
duration  in  C2NET) ;  most  of  it  will  not  be  simple.  The  next 
chapter  discusses  recommendations  and  potential  problems  in 
modifying  C2FAM  (that  is,  C2NET,  the  force  evaluation  model, 
and  linkage  mechanisms)  in  accordance  with  this  proposal. 
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IV.  AREAS  FOR  FUTURE  STUDY 


A.  INPUT  PROCESS  VALIDATION 

From  our  analysis  of  C2NET  we  know  there  is  much  work  to 
be  done  to  improve  the  input  process.  This  section  identifies 
areas  of  potential  trouble  to  Army  analysts  in  implementing 
this  proposal  and  recommends  procedures  to  solve  these 
problems . 

The  key  to  linking  the  tactical  scenario  of  the  evaluation 
model  into  C2NET  is  the  concept  of  operational  tempo. 
Creating  these  OPTEMPO  timelines  for  each  scenario  of  interest 
will  challenge  the  analyst  due  to  the  number  required  (one  for 
each  unit  by  operating  system)  and  lack  of  clear  definitions 
for  "High, "  "Medium, "  and  "Low"  OPTEMPO  for  each  battlefield 
operating  system  (BOS) .  A  simple  heuristic  approach  to 
constructing  these  timelines  is  based  on  the  concepts  of  Air- 
Land  Operations  (ALO) .  ALO  identifies  four  stages  of 
tactical/operational  combat: 


•  Stage  1 

•  Stage  2 

•  Stage  3 

•  Stage  4 


Detection  and  Preparation  Phase 

Establish  Conditions  for  Decisive  Operations 

Decisive  Operations 

Reconstitution 
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We  state  without  proof  that  OPTEMPO  for  each  BOS  is 


dependent  on  the  ALO  stage  a  unit  is  in  (e.g.,  a  unit 
conducting  decisive  operations  would  have  a  higher  maneuver 
tempo  than  a  unit  undergoing  reconstitution)  .  We  can 
therefore  approximate  all  BOS  OPTEMPO  timelines  if  we  know 
when  a  unit  is  in  which  ALO  stage  and  the  corresponding 
OPTEMPO  values  for  each  stage.  This  approximation  will 
simplify  the  timeline  construction  process,  as  the  scenario 
developers  know  which  stage  each  unit  is  in  for  a  scripted 
model  (like  VIC) .  Table  5  indicates  how  BOS  tempo  may  be 
approximated  by  ALO  stage. 

TABLE  5.  BOS  OPTEMPO  BY  ALO  STAGE10 


f 

£ 

MVR 

INTEL 

FIRE 

SPT 

CSS 

AIR 

DEFENSE 

i 

L 

H 

M 

L 

l 

i 

L  i 

2 

M 

M 

M 

L 

1 

M 

3 

H 

H 

H 

M 

H  j 

4 

M 

L 

L 

H 

M 

.  . 

10  Combat  Service  Support  (CSS)  and  Air  Defense  are  new 
modules  under  development  for  incorporation  into  C2NET  during 
FY93 . 


The  other  major  obstacle  to  our  input  process  improvement 
effort  is  database  validation.  As  demonstrated  in  our 
hypothesis  testing,  the  subject  matter  expert  estimates  for 
the  arrival  process  are  not  necessarily  accurate  and  there 
remains  some  question  as  to  the  proper  parameter  to  use  in  the 
service  process.  However,  now  that  we  are  confident  we  have 
identified  the  correct  underlying  theoretical  distributions, 
it  is  possible  to  test  the  SME  estimates  against  observed  data 
of  the  real  system. 

Just  as  model  validation  is  an  ongoing  process,  so  should 
database  validation  be  a  continuous  effort.  Army  analysts 
involved  with  C2NET  should  continue  to  gather  test  data  from 
other  analytical  agencies  (e.g.,  the  ARI  studies  and  Force 
Level  Control  System  experiments  conducted  by  the  MITRE 
Corporation) ,  screen  these  data  for  applicability  to  C2NET 
tasks,  and  then  conduct  hypothesis  testing  on  the  database 
parameters.  Prior  to  testing,  database  parameters  can  be  used 
in  their  current  form,  with  the  caveat  that  C2NET  results  are 
only  valid  in  use  with  comparative  studies,  where  database 
errors  will  tend  to  have  equal  effects. 

B.  OUTPUT  PROCESS  VALIDATION 

In  developing  C2  effectiveness  inputs  to  the  evaluation 
model  the  analyst  must  resist  the  temptation  to  go  directly 
from  identification  of  key  explanatory  variables  to 
construction  of  a  response  table.  The  approach  discussed  in 
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Chapter  III  assures  the  analyst  that  the  effects  of  key 
variables  are  correctly  defined  and  the  results  input  into  the 
evaluation  model  are  accurate  approximations  of  C2 
effectiveness . 

The  most  significant  challenge  to  the  analyst  in  this 
process  is  correctly  constructing  the  fuzzy  set(s)  with 
corresponding  restrictions  and  penalties.  Subject  matter 
experts  should  be  involved  in  this  process,  using  their 
empirical  knowledge  of  the  system  to  map  the  degrees  of 
membership  for  the  base  variables  into  the  fuzzy  restrictions. 
In  the  artillery  example,  it  would  be  proper  to  ask  artillery 
instructors  at  the  Command  and  General  Staff  College,  based  on 
their  personal  observations,  questions  relating  to  mission 
effectiveness  given  processing  delay  times  and  target  speed. 
Their  answers  to  these  questions  could  then  be  used  to 
construct  SME  estimates  of  the  fuzzy  sets.  Where  possible, 
these  estimates  should  then  be  tested  against  real  system 
performance  data  (e.g.,  National  Training  Center  observations, 
Desert  Storm, . . . ) . 

The  final  step  in  our  approach,  inputting  the  response 
table  into  the  evaluation  model  and  running  the  model,  is  also 
a  challenge.  As  a  scripted  model,  changing  the  attributes  of 
the  model  (attrition  coefficients,  maneuver  paths,  resupply 
rates,  ...)  can  have  unexpected  consequences.  Additionally, 
enemy  forces  operating  without  penalties  (from  less  than 
perfect  C2)  will  have  an  unrealistic  advantage. 
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To  reduce  the  inpact  of  the  first  problem,  the  analyst  in 
conjunction  with  the  scenario  developer,  will  need  to  conduct 
an  analysis  of  the  model  after  C2  response  tables  are  input. 
This  will  initially  represent  significant  work,  as  the 
penalties  may  need  adjustment  to  better  approximate  C2  effects 
in  the  model.  The  second  problem  will  require  either 
development  of  a  Threat  C2NET  or  some  more  generalized 
methodology  to  account  for  C2  effects  on  the  operational 
capabilities  of  enemy  forces. 

C.  CONCLUSIONS 

The  intent  of  this  proposal  is  to  identify  significant 
disparities  between  the  Command  and  Control  Functional  Area 
Model  and  the  real  world  performance  and  effects  of  command 
and  control  systems,  and  recommend  potential  solutions  to 
reduce  or  eliminate  these  disparities.  Through  examination  of 
the  C2FAM  input  and  output  processes,  this  thesis  has  met  its 
goal.  However,  even  with  successful  implementation  of  all  the 
recommendations  as  proposed,  the  validation  process  for  C2FAM 
will  not  be  completed.  New  command  and  control  systems  and 
changes  in  warfighting  concepts  will  require  a  continuous 
effort  by  Army  analysts  to  insure  that  the  C2FAM  (C2NET, 
evaluation  model,  and  linkage  mechanisms)  remains  a  valid, 
worthwhile  model  into  the  future. 
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APPENDIX  A.  BASICS  OP  C2NET 

Thi  command  and  control  network  model  (C2NET)  is  a  low 
resolution  model  representing  essential  command  and  control 
(C2)  tasks  performed  by  Corps,  Division,  and  Brigade  command, 
control,  communications,  and  intelligence  (C3I)  systems 
(Command  Posts,  Fire  Direction  Centers,  Intel  Fusion  Centers, 
etc.)  .  C2NET  was  developed  as  a  direct  result  of  the  need  for 
a  command  and  control  functional  area  model  (C2FAM)  to  assist 
Army  analysts  in  conducting  trade-off  analyses  as  fiscal 
constraints  grow  tighter,  as  well  as  explore  new  emerging 
warfighting  concepts.  Output  (measures  of  performance)  from 
the  C2NET  performance  model  is  intended  to  be  used  as  C2  input 
parameters  for  force  effectiveness  models.  To  date,  C2NET 
output  has  been  used  with  the  Vector- in- Commander  (VIC) 
effectiveness  model  to  study  C2  responsiveness  and  alternative 
Division  command  post  configurations. 

C2NET  was  built  in  FY91  by  Potomac  Systems  Engineering 
(PSE)  for  the  TRADOC  Analysis  Command  -  Operations  Analysis 
Center  (TRAC-OAC) ,  Fort  Leavenworth,  KS.  C2NET  is  based  on 
the  Modeler  programming  tool .  Modeler  is  an  interactive  tool 
written  in  the  "C"  programming  language  that  uses  a  graphical 
interface  to  produce  Stochastically  Timed  Attributed  Petri  Net 
(STAPN)  event-stepped  simulation  models.  A  Petri  Net  is  a 
bipartite  graph  consisting  of  queues  and  servers  connected  by 
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arcs.  Petri  Nets  also  employ  primitive  symbols  and  objects 
known  as  "tokens"  (customers)  that  accumulate  in  queues  and 
are  consumed  when  served. 

For  descriptive  purposes,  the  Petri  Net  (C2NET)  closely 
resembles  an  Open  Queuing  Network,  complicated  by: 

•  Before  service  can  occur,  all  queues  into  the  server  must 
be  occupied. 

•  Customer  generation  can  occur  internal  to  the  network 
(each  service  can  generate  more  than  one  customer,  with 
each  arc  from  server  to  queue  generating  a  customer) . 

Despite  these  caveats,  many  of  the  basic  modelling 
considerations  of  an  open  queuing  network  apply  to  C2NET,  and 
must  be  addressed  before  C2NET  is  validated  (such  ar 
interarrival  rates  [task  generation] ,  service  rates,  and 
corresponding  theoretical  distributions) . 

The  Modeler  programming  tool  allows  the  C2NET  post¬ 
processor  to  produce  output  using  four  measurable  quantities: 

•  Probability.  The  model  probabilities  for  each  flow  of 
tokens  across  the  arcs  connecting  each  server  (or  queue) . 

•  Rate.  Transition  frequencies  from  place  to  place  during 
the  simulation. 

•  Occupancy.  The  average  number  of  tokens  in  each  node 
during  the  simulation. 

•  Delay.  The  time  between  transitions. 


The  key  tasks  currently  represented  in  C2NET  include  those 
from  the  maneuver  (MVR) ,  intelligence  and  electronic  warfare 
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(IEW) ,  and  fire  support  (FS)  battlefield  operating  systems. 
The  combat  service  support  and  air  defense  operating  systems 
are  under  study  for  planned  incorporation  into  C2NET.  A  list 
of  key  tasks  currently  modeled  (by  echelon  and  BOS)  is  at 
Table  6.  Additional  information  may  be  obtained  regarding 
C2NET  by  writing  the  Director,  Command  and  Control, 
Communications,  and  Intelligence  Studies  and  Analysis 
Directorate  (C3I  SAD),  ATTN:  ATRC-FS,  Fort  Leavenworth,  KS, 
66027-5200. 
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TABLE  6.  C2NET  KEY  TASK  LIST 


ECHELON  MODELED: 


KEY  TASKS 

C 

■a 

B 

MANEUVER  TASKS 

Assess  Tactical  Operations 

X 

X 

Develop  Attack  Criteria 

X 

X 

X 

Develop  Commander's  Planning  Guidance 

X 

X 

X 

Develop  Concept  of  Deep  Operations 

X 

Develop  Engineer  Priorities 

X 

X 

X 

Develop  Force  Alternative  Courses  of  Action 

X 

■ 

Develop  Subordinate  Command  Missions 

X 

X 

3 

Evaluate  Unit  Status 

X 

X 

H 

Prepare  FRAGOs 

X 

X 

X 

Prepare  Operational  Orders 

X 

X 

Prepare  Operations  SITREP 

X 

X 

■ 

Prepare  Warning  Order 

X 

X 

mm 

Process  FRAGO 

X 

X 

Select  Force  Course  of  Action 

X 

X 

X 

Update  Status  of  Adjacent  Forces 

X 

Update  Status  of  C3CM  Situation 

X 

Update  Status  of  Mobility/Counter-Mob  Situation 

X 

|  Update  Status  of  Engineer  Situation 

X 

|  Update  Status  of  Friendly  Forces 

X 

■9 

X 

Update  Status  of  OPSEC  Situation 

X 

X 

Update  Status  of  Plans  and  Orders 

X 

Update  Status  of  Tactical  Situation 

X 

X 

X 
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TABLE  6.  C2NET  KEY  TASK  LIST  (CONTINUED) 


FIRE  SUPPORT  TASKS 

Coordinate  Army  Aviation  Support  (CAS) 

X 

X 

Coordinate  JAAT  for  Close  Operations 

X 

Coordinate  Tactical  Air  Support 

X 

X 

X 

Develop  Priority  of  Fires 

X 

X 

X 

Process  Requests  for  Aviation  Support 

X 

X 

X 

Process  Requests  for  Close  Air  Support 

X 

X 

X 

Process  Requests  for  Immediate  Artillery  Support 

X 

X 

Process  Requests  for  Immediate  MLRS  Support 

X 

X 

Process  Target  Nomination  (FS) 

n 

X 

X 

Process  Target  Nomination  (Sensor) 

X 

X 

X 

Update  Status  of  Air  Mission  Results 

X 

X 

■ 

Update  Status  of  Air  Support  Assets 

X 

X 

Update  Status  of  Artillery  Units 

X 

X 

X 

Update  Status  of  MLRS  Units 

X 

Update  Status  of  Conventional  Targets 

X 

n 

X 

Update  Status  of  Conventional  Fire  Requests 

X 

Update  Status  of  Conventional  Strike  Results 

X 

Update  Status  of  FS  Order  of  Battle 

X 

Update  Status  of  FS  Situation 

X 

X 

X 

Update  Status  of  JAAT  Operations 

X 

X 

X 

Update  Status  of  Joint  Air  Support  Operations 

X 

X 

INTELLIGENCE  &  ELECTRONIC  WARFARE  TASKS 

Process  Critical  Deep  Operations  Events 

X 

Process  Critical  Situation  Reports 

X 

X 

X 

Process  Intelligence  Spot  Report 

X 

X 

X 

Update  Status  of  Current  Intelligence 

X 

X 

X 

Update  Status  of  Deep  Operations  Situation 

X 

Update  Status  of  Target  Acquisition  Capability 

X 

X 
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PCSM 


Figure  18.  PCS  Hid  OPTSKPO  [Exp  I  #-15)] 
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EXPONENTIAL  DENSITY  FUNCTION.  N»67 
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PCSH 


EXPONENTIAL  DENSITY  FUNCTION.  N-34  EXPONENTIAL  CUMULATIVE  DISTRIBUTION  FUNCTION. 


Figur*  20. 


PIS  Low  OFTEKPO  [Sap (0-60)] 
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PISL 


EXPONENTIAL  DENSITY  FUNCTION.  N«18  EXPONENTIAL  CUMULATIVE  DISTRIBUTION  FUNCTION, 
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GAMMA  DENSITY  FUNCTION,  N-31  GAMMA  CUMULATIVE  DISTRIBUTION  FUNCTION, 


EXPONENTIAL  DENSITY  FUNCTION,  N-24  EXPONENTIAL  CUMULATIVE  DISTRIBUTION  FUNCTION,  N-24 


'll 
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GAMMA  DENSITY  FUNCTION, 


» 

Figure  25.  UFF  Mid  OPTBMPO  [Gamma (er-2,  fim FREE)] 
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UFFV 


EXPONENTIAL  DENSITY  FUNCTION.  N-130 


•*  •»  m 


ADN3H038J  3A11VT38  AinteVBOHd  3AU\nfWnD 

Figure  27.  PCS  Task  Durations  [Exp (0-2)] 
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TASK  DURATION  (MINUTES) 
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Figure  28.  UFF  Teals  Durations  [Exp (0-3)] 
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